Design and Implementation of Feed Forward Neural Network Based Chatbot for Depression Support

Abstract

As the awareness related to mental health is increasing, there is a need to make the assistance related to it more accessible. According to recent studies, people avoid in person counselling when it comes to mental health. Hence alternatives need to be made available to help people to live a stress-free life. We proposed this system in hope to address this topic in form of a chat-bot which engages users to have a positive conversation. This will surely reduce the loneliness and stress which people might find difficult to share with other people. This chatbot is built using feed forward neural network to generate appropriate response.

Introduction

I. INTRODUCTION

Each year, an estimated more than 40,000 adolescents die by their own hands, a statistic that UNICEF raises concern over in its annual report since it reflects the fact that one of small group of people battle with their mental health. One of the key causes is the increase in social media usage. Live in virtual life more than practical life. They can only hurt each other in a very small number of ways that seriously harm their development, self-esteem, and cause worry, social anxiety, despair, and even suicide. Social media has contributed to an upsurge in bullying, which has virtually little parental or adult supervision. In this age of competitiveness, it can be challenging for age of 15 years to 45 or more years people.

Chat bot that communicates with teens via their smart phones is a simple and practical way to assist people who are in need and youths. Instead of stand in waiting lists for appointments with psychologists continue to lengthen, chatbots can assist an infinite number of teenagers right away. It’s like having something readily available in their surroundings that they can quickly access, talk with them, share their feeling.  Although it may seem contradictory, using smart devices to address teenagers’ anxiety caused by the same screens works. It’s a really, really good idea that might really benefit kids who don’t feel like they have a someone like parent or friend to talk to or who don’t have those kinds of relationships. Cell phones are now practically universal among teenagers. Talking to a robot is more comforting than having a face to-face chat since it cannot make individuals feel bad about themselves because that is not its intended purpose.

II. LITERATURE SURVEY

With rising mental health related issues, it is important to fight depression by leveraging technology by using various technologies available. Though machines cannot be as intelligent as humans the can definitely automate a part of anti-depression therapy and can be useful in various ways and people are find it interesting to share their problems with an chatbot[1].

Many chatbots have been developed some of milestones are: In 1950, Alan Turing proposed turing test which made everyone wonder if machine can talk to humans without making them realize that they are talking with a machine[2]. ELIZA was the first bot developed fourteen years later which was based on pattern matching which worked on already feeded responses and matched keywords and therefore had limited capabilities[3]. Jabberwacky introduced in 1988 was developed using CleverScript but was slow and not being able to handle multiple users[4]. Dr. Sbaitso (i.e. Sound BlasterActing Intelligent Text to Speech Operator) was the first psychologist chatbot but was unable to make deep conversations[5]. ALICE was another example of pattern matching chatbot developed using AIML(Artificial Intelligence Markup Language)[6]. Google Assistant, Apple Siri, Amazon Alexa, Microsoft Cortana and IBM Watson are the examples of modern artificial intelligence based chatbots which can be easily accesed through mobile devices and are very powerful.

As need of mental health chatbots have never been more before, many mental health chatbots are developed some of them are: Woebot[7] keeps track of mental health of user and gives cognitive behavioural therapy to user on regular basis. Wysa[8] in addition to various techniques promotes meditation, motivational interviewing, etc to combat depression. Tess[9] tries to imitate humans to act as partner with whom we can share our feelings uses various therapy techniques, gives personalised solutions and is based on decade of research work.

Chatbots can be built using various natural language processing techniques like Term Frequency-Inverse Document Frequency (TF-IDF), Word2Vec, Stemming, Removing Punctuation Marks and Stop words, Tokenization, Intent Classification, Word Embedding, etc[10].

III. IMPLEMENTATION

A framework for a chatbot that defines conversational intents is necessary. JSON files offer a neat solution for this. Each conversational intent has a tag (unique identifier) for our neural network text classifier, patterns (sentence patterns) and responses (one will be used as a response) we’ll also subsequently include some fundamental contextual components. We dealt with our imports first. We can now start organising our documents, words, and classification classes after loading our intents. JSON file. Each sentence is made up of a list of words that have been stemmed, and each document is linked to a certain intent (a class). Data conversion from string to vector is necessary for training. We divide our intentions and words into an array with the use of a bag of words. Both “bag of words” arrays for the intent pattern and the intent class are visible.

In Feed forward network the inputs are bag of words, input layer and output layer contain number of pattern and number of classes. In next step some NLP techniques are used such as tokenization and stemming they help in excluding punctuation characters. Natural language tool kit framework used to download packages such as pytorch. Pytorch is an open-source machine learning framework that accelerates the path from research prototyping to production deployment. Next step is to create a stemmer and test the stemming. After tokenising the sentences, we head back to the training file and then created a pytorch dataset. Next step is to create actual model, for this we create a new file then import torch and neural network creation of new class for model include self, input size, hidden size. Then we applied SoftMax for each classes which contain three linear layer. Each layer has input size and hidden size, then it also creates a activation function ReLu. From model import neural Net , it gets input size, hidden size and output size. After creation of pytorch model and training pipeline next part is Save/load model and implement the chat.

At the end from the training file to save the dictionary is created. Model state, input size, output size and hidden size all saved in the dictionary and also stored the words collected in the beginning, torch. Save(FILE) to save the data. After completion of training and saving of file only thing left is to implement the actual chat. So trained another file called chat.py and import random,json,torch. Checking of GPU support is also a much-needed step here. Then we opened the intent file and our saved file, saved all words and model state. To load the model after creation it knows our learned parameters and set it to evaluation mode. Check for implementation and create a bot with a name and using while loop check for input condition. We need to tokenise the sentence, create bag of words and reshape it. By using model as output which gives prediction. At the end we check for the probability and SoftMax.

IV. TRAINING RESULTS

Following figure shows how model trained, 150 epochs are enough for the dataset used as loss is reduced to 1 percent.

VI. FUTURE SCOPE

Following are some of the advancements that can be implemented to make this chatbot better.

1. Advanced NLP - The NLP (Natural Language Processing) used in our chat-bot is done through the in-built functions from nltk library which are quite simple and may lower the accuracy of the results. We can design our own processing function to boost the accuracy.
2. Integration with devices - Integration with handled and mobile devices will increase the reach and ease of use of the chat-bot. These will include smart watches, mobile phones etc.
3. Professional health care support - Collaborating with health care professionals will make the application trust-worthy and errors resisting.

Conclusion

We have developed a web-based chat-bot to contribute to the growing concern of stress and mental health issue. We have used natural language processing and neural networks to implement a chat-bot which have the potential to lower stress and anxiety by engaging user in a conversation and providing positive replies and suggestion. Although in-person counselling always has an upper hand over these AI conversations, we have referred various counselling conversations by professionals to make the chat-bot as realistic as possible. We have tested the chat-bot through various types of inputs, and it was able to handle light severe type of cases. We have also tried to make the UI interactive and simple.

References

[1] Hill, J., Randolph Ford, W., & Farreras, I. G. (2015). Real conversations with artificial intelligence: A comparison between human-human online conversations and human-chatbot conversations. Computers in Human Behavior, 49, 245–250. https://doi.org/10.1016/j.chb.2015.02.026. [2] Turing, A. M. (1950). MIND A QUARTERLY REVIEW OF PSYCHOLOGY AND PHILOSOPHY I.COMPUTING MACHINERY AND INTELLIGENCE (Issue 236). [3] Shawar, B. A., & Atwell, E. S. (2005). Using corpora in machinelearning chatbot systems. International Journal of Corpus Linguistics, 10(4), 489–516. doi:10.1075/ijcl.10.4.06sha [4] Wikipedia(2022) Retrieved from https://en.wikipedia.org/w/index.php?title=Jabberwacky&oldid=926273345 [5] Wikipedia(2022) https://en.wikipedia.org/wiki/Dr. Sbaitso [6] Wallace, R.S. (2009). The Anatomy of A.L.I.C.E.. In: Epstein, R., Roberts, G., Beber, G. (eds) Parsing the Turing Test. Springer, Dordrecht. https://doi.org/10.1007/978-1-4020-6710-5 13 [7] Prochaska J, Vogel E, Chieng A, Kendra M, Baiocchi M, Pajarito S, Robinson A, A Therapeutic Relational Agent for Reducing Problematic Substance Use (Woebot): Development and Usability Study J Med Internet Res 2021;23(3):e24850 URL: https://www.jmir.org/2021/3/e24850 DOI: 10.2196/24850 [8] Inkster B, Sarda S, Subramanian V, An Empathy-Driven, Conversational Artificial Intelligence Agent (Wysa) for Digital Mental Well-Being: Real-World Data Evaluation MixedMethods Study JMIR Mhealth Uhealth 2018;6(11):e12106 URL:https://mhealth.jmir.org/2018/11/e12106 DOI: 10.2196/12106 [9] Fulmer R, Joerin A, Gentile B, Lakerink L, Rauws M, Using Psychological Artificial Intelligence (Tess) to Relieve Symptoms of Depression and Anxiety: Randomized Controlled Trial JMIR Ment Health 2018;5(4):e64 URL: https://mental.jmir.org/2018/4/e64 DOI: 10.2196/mental.9782 [10] Manaswi N. K, (2018). Developing Chatbots,Deep Learning with Applications Using Python : Chatbots and Face, Object, and Speech Recognition With TensorFlow and Keras(145–170). DOI:10.1007/978-1-4842-3516-4 11

Copyright

Copyright © 2023 Pankaj Atram, Swapnil Chitalkar, Faizan Ali, Kedar Muluk, Amit Kadam. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.